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ABSTRACT 

Creativity is an important ability in teaching-learning process because of its contribution to the struggle to cope 
with complex problems, to ensuring satisfaction in life and to the improvement in professional development. 
Especially, creativity is one of the essential abilities that teachers need in teaching-learning processes in their 
classrooms. It is crucial that prospective teachers’ creative thinking skills should be developed and prospective 
teachers produce teaching materials in a creative way. 

This study aims to develop a scale for the evaluation of the materials, which are developed by hand, or as 
technology-based, and of the materials, which are developed as project-based by prospective teachers in 
Instructional Technologies and Material Development (ITMD) Course or in other courses, in terms of creativity. 
First, the literature has been reviewed to determine the items that will take place in the 5-likert type scale, and a 
pool of items has been developed by the researchers. 

A draft questionnaire has been prepared with the arrangement of the items created out of the pool with 5-likert 
type scale from strongly disagree to strongly agree statements and factor scores have been tried to be explained. 
The creativity of the materials is investigated in four categories- fluency, flexibility, originality, and elaboration. 
This questionnaire has been presented to the experts studying on creativity and material development in faculties 
of education to get their opinions. Items in the scale have been determined in the light of the results of the data. 
At the second stage, 4 experts have evaluated sample materials by the use of this scale, and reliability analysis 
has been carried out through the explanations for the harmony levels of these four experts. 

Key Words: Creativity, Material, Instructional Technologies and Material Development. 

1. INTRODUCTION 

Defining the concept of creativity is difficult but it is one of the most charming concepts. Creativity is the 
discovery of the essence in every individual potential. It can be seen in a work of Leonardo da Vinci as well as in 
a housewife’s home decoration. Being peculiar, inventing, multi-dimensional consideration, fluency in 
consideration and redefining are of the concepts that form creativity. 

Creativity is an ability that can show itself in every part in human life and exist on every level. It exists from 
daily life to academic studies, it is a whole process causing of masterpieces in artistic area and also is an attitude 
and behaviour style (San, 1993). As David Bohm (1998) described it in his book named “On Creativity” as 
“according to me it is difficult to describe it”, creativity is represented in different discipline areas differently. 
Creativity is described in the area of training as “discovery”, in business world as “enterprise”, in the area of 
mathematics as “solving problem”, in the area of music as “performance” or “composition” (Reid & Petocz, 
2004). 

Although it is difficult to provide an agreement in the definitions that are introduced or to make a scientific 
definition, every definition is important due to bringing a different point of view to creativity. The emergence of 
creativity is generally considered and denoted by predicating on definition models that determine the boundaries 
and give denotation. 

1.1. Creativity 

Creativity is one of the important skills that human beings maintained interest on for a long time. This skill 
existing in every individual to some extent (more or less) is an important requirement from the social aspect 
together with the addition to individual life such as overcoming complex problems of life, providing satisfaction 
from life and self realizing. It is an undeniable truth that creative discoveries have an important place in 
developments of a society. 
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Researchers, who emphasize creativity a concept that should be considered at least at four-dimensional, specified 
that creativity can gain a conceptual integrity when the dimensions organized as a creative person, creative 
process, creative product and creative press are enlightened entirely (Money, 1963; MacKinnon, 1970; 
Woodman & Schoenfeld, 1989). In order to prove himself, creativity is a potential power that should be in the 
individual when encountered the suitable situation and exists in everyone more or less even if he is artist or 
scientist (Rouquette, 1992). A creative person is an individual who can find new solutions for problems and 
make a synthesis at complex and new level (San, 1979). 

Self confidence, taking risk, being enterprising, being ambitious, high energy and being adventurer, curiosity, 
comfortable consideration, being diffirent, tolerance of mistakes, flexible and racial consideration, are some 
features that a creative person should have (Mac Kinnon, 1970; Ank, 1987; Rouquette, 1992; Zuckerman, 1979). 
Creative person investigates new areas, makes new observations, makes new predictions and makes new 
inferences (Amabile, 1983, 1996; Rtza, 1999; 2002; Williamson, 2001). According to National Advisory 
Committee on Creativity and Cultural Education (1999; 29), “creativitity is an imaginative activity fashioned so 
as to produce outcomes that are both original and value”. 

Creative press is related with the nature of the problem, physical environment and familiar environment. The 
uniqueness of the problem, the aptitude of the person to solve the problem, complexity, dimension and process 
are situational factors (Amabile, 1983; Torrance, 1995). The perception of the problem, the ability of describing 
the work more or less such as easy or complex, finding solution, finding different ideas, explaining only good 
ideas, make the right selection between the strategies, to dedicate oneself to working shows that creative 
situation exist in getting acquainted with the environment. 

Some researchers consider creativity as a process. Wallas (1926) describes creativity as a process which is 
realized in four steps like; preaparation, incubation, illumination and verification. Torrance (1995: 23) also 
describes creativity as “the process of forming ideas or hypotheses, testing hypotheses and communicating the 
results”. Swede (1993) describes creativity as a process of at least carrying two features such as ‘unique’ and 
‘valuable’ rather than finding an answer to a question. 

As it is stated before, creativity depends on innovation and discovered things different from individual, situation 
and process (Arik, 1987; Olson, 1999). Meanwile a distinction is usually made between the creative product and 
the creative process (Riza, 2002; Russ, 2003). The creative product is the output of the individual, which can be 
judged as to the amount of creativity. If the product is new, suitable, useful and true or is valuable in useful, it 
can be defined as creative (Amabile, 1983; 1996; Glover, Ronning & Reynolds, 1989; Tezci, 2002). Creativity is 
the capacity of man to produce new ideas, opinions, discoveries or artistic objects which are valuable for social, 
moral, esthetic, scientific or technological use (Arik, 1987: 226) and also creativity necessitates reaching a new 
synthesis, to find new solutions and to display new and original products (Demirel, 2003: 226). Not being 
estimated before and being really original, being different from works which have been produced before by other 
people and confusing lots of people are the features that a new product or idea should consist (Lubart, 1994; 
Giirol & Tezci, 2001; Dikici, 2001; Yanpar, Koray, Parmaksiz ve Arslan, 2004). 

Although the general features of creativity and the process differ from person to person there are some common 
features that define creativity. They are fluency, flexibility, originality and elaboration as follows (Guilford, 
1950; Torrance, 1968, 1974; Amabile, 1983; Weisburg, 1986; Paulus, 2000; Kincaid & Duffus, 2004): 

Fluency: Fluency is the ability to generate a large number of ideas, consequences or possibilities, and having the 
ability to produce different ideas and hypothesis in related to the problems that engage the mind of a person. 
Fluency is the ability to produce lots of ideas in different dimensions and the ability to choose the valuable ones 
of these ideas for a specific purpose. Fluency is arranging the works which are various and detailed. Word 
fluency can be classified as an association that can be described as finding the synonyms of given words and is 
related with semantic ability, expression fluency that is the ability of express the meaning with different word 
groups and sentences, mental fluency that is the ability to produce ideas in order to provide some requirements 
(Guilford, 1950; Torrance, 1968; Kincaid & Duffus, 2004). 

Flexibility: Flexibility is ability to generate a vide variety of ideas, and adapting to changing conditions, 
thinking independently, and creating different aspects. It is thinking and producing without staying stable, unlike 
others. It is an ability of changing point of view, redefining problems by making more concrete and abstract 
when necessary. Spontaneous flexibility is defined as thinking independently although it is not necessary. 
Individual, thinking creatively, can quickly pass from one category to another. For example, he can think the 
brick, a building material, as a weighing tool or rocket. Adapter flexibility is useful for problems that need 
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unusual sollutions. Sometimes problems seem to be able to be solved by traditional methods. But these methods 
cannot be used for each problem (Torrance, 1968, 1974; Amabile, 1983; Weisburg, 1986) 

Originality: It is defined as producing extraordinary answers, being original in thoughts and actions, and 
sometimes breaking taboos of community and evading regulations. It expresses individualism, uniqueness, and 
quite dissimilarity. It also expresses being unique, being new, and presenting the best. It requires being different 
or being unusual, unlike others. Obvious responses are not considered original. It expresses forms that others 
could not do and could not reach in advance (Amabile, 1983; Weisburg, 1986; Paulus, 2000; Kincaid & Duffus, 
2004) 

Elaboration: It expresses to get down to the details of suggested opinions and to be detailed in opinions, 
thoughts and actions (Sungur, 1992; Yanpar et al., 2006). Elaboration provides depth of thoughts. It is reflected 
multiple responses involving of detail. 

1.2. Teaching Materials 

Modern life makes individuals to use their creative thinking skills for solving social, individual and professional 
problems in a high value appearing complicated, and to bring up creative products. Although communities aim 
to raise individuals that harmonize with their prejudices, laws and values, it is important to improve creativity 
from science to education system in each field in rapidly changing world. Teachers have an effective role on 
students to reveal and improve their creative potential. Teachers have to be creative in many subjects like 
choosing different education approaches, using different kinds of techniques and methods together related to 
their theme, choosing and preparing required materials. For this reason it is important for teachers to get 
students’ creative skills improved, and educate them in way of preparing creative materials to study in education 
faculties. Especially nowadays when creativity is important required skills for teachers in teaching-learning 
process, it is important for prospective teachers who study in faculties of education both to improve their creative 
thinking skills and to develop education materials related to their areas in a creative way before appointed as 
teachers. 

Education materials, in general meaning, are every kind of materials that are used making teaching-learning 
process effective (Yildiz, 2004). The purpose of using of materials is sometimes modeling among sub-themes 
related to main themes, sometimes activating learning, sometimes concretizing hardly understandable themes, 
depicting, etc. So every kind of supportive object towards activating education and improving productivity can 
be called as educational materials. Material that is used for any action can be a symbolic system that serves 
emphasizing and conveying meanings (Goodman, 1978) or it can be a computer (Salomon, 1993) program that 
makes possible to create and manipulate mental strategy or symbolic and mental objects that aim to trasfer 
information to others as well The important thing is to make synthesis for materials in a creative way for students 
to improve their acquisition in courses. While making this synthesis, one should act in frame of a forementioned 
collective features that indicate creativity. Prospective teachers should be independent in process of forming 
creative materials (Yanpar, et al., 2006). He should also think by making connections, should express his 
thoughts easily, should examine, research and critics. 

1.3. Rubric 

Rubric, one of the authentic evaluation tools, can be used for evaluating formed materials. Rubric is a tool that 
consists of criterions that define student’s performance, and is useful for evaluating performance in different 
levels with these criterions. It consists of criterions that were developed for evaluating students’ examinations, 
folders, homework and performances. According to Goodrich (1997) rubric, as being an evaluational tool, is 
listing criterions for a piece of work. Gronlund (1998) defines rubric as an evaluational instructions or 
evaluational guide that defines characteristic features about a certain subject related to performance in different 
levels and is used for deciding related to performance. 

Rubric can be used for evaluating actions such as oral projects (class discussion, acting, interview, oral 
representations, story telling, debate, etc.), products (collection or exhibition, preparing brochure, preparing 
poster, publishing newspaper, etc.), experiment reports, drawing graphic, solving problem, preparing project, 
researching and writing studies (writing letter, petition, story or composing), artistic studies (Tezci, 2005; 
Kubiszyn & Borich, 2003; Gronlund, 1998). 

Rubric can be used in different kinds of performance or success areas. For example, it can be used in products 
that include poet, article, graphic, exhibition, picture, photograph; in mental processes such as organizing and 
using ability; in noticeable performances like typing computer, playing an instrument, oral explanation, using a 
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tool; and in behavior and social skills like mental practices, group studying skills, and self recognition (Kubiszyn 
& Borich, 2003; Danielson & Abrutyn, 1997). 

The aim of the study is to develop a scale for the evaluation of the materials prepared manually or technology- 
based and developed as project-based in point of creativity in “Instructional Technologies and Material 
Development” course or in other courses. 

2. METHODOLOGY 

2.1. Development of the scoring guideline 

In this section, explanations about the process of preparing the teaching materials and the analysis during this 
process are discussed. 

2.1.1. Identification of the dimensions that is going to take place in the scale 

This step is a phase in which what kind of things will be in the content of the study are determined. Grading 
rating scale should specify what the students should know, think and do; and it is also a phase to determine what 
the performance will look like. The definitions in the content should involve clear and familiar explanations. 
Relativity should be minimized (relativity should be lowered to a minimum level) (Wolf, 1999; Aschbacher, 
Koency & Schacter, 1995; Burstein, Koretz, Linn, Sugrue, Novak, Baker & Lewis, 1996). 

In the grading rating key, the content should have the qualification of differentiating a well-qualified study from 
a less qualified study. The validity of the rating tools that are used in evaluating the studies is the process of 
increasing the evidences which will suplement the suitability of the results inferred from the reactions such as a 
specific task, assignment and a study (Marzano, 1996; Moskal & Leydens, 2000). As the validity depends on the 
purpose of the evaluation, what is going to be put forth from the reactions of the students should be defined 
clearly and appropriately. In this context, what should take place in the content of the scale is determined in the 
first step. 

The scale was started to be done firstly by finding out the indicators of performance that is appropriate for the 
criterion such as fluency, originality and enrichment of the material. In order to find out the structure of the 
scale, firstly, 63 items was prepared as a result of a literature analysis. It was given great importance to the fact 
that the features such as creativity, fluency, flexibility, originality and enrichment dimensions were handled all in 
one or at least one- two features were mentioned together. 5 point-likert type scale (ranging from 5= strongly 
agree to 1= strongly disagree) which consists of 50 items was prepared by removing 13 materials totally. 2 of 
these materials were found out to have no relation with the creative material after it was undergone a specialist’s 
judgement and the remained 11 items seemed to be the repetition of the materials. After marking, the extracted 
materials were put into the scale again in order to keep control over the application phase of the scale. By in 
terms of the difficulty in writing the adequate controverse expressions in the scale and the fact that it is not 
suitable to write negative expressions in the grading rating scale, this method was used (Finson & Ormsbess, 
1998; Goodrich, 1997; Burstein et al., 1996). 

The prepared scale was applied to 112 specialists in both the field of material development and creativity (they 
give “teaching Technologies and material development” courses at the ducation faculties and they carry their 
studies on creativity). Factor analysis was applied to the data gathered by means of the scale, which has been 
applied. Factor analysis is a procedure that tries, by combining numerious variables, to find few new variables, 
which are unrelated and conceptually meaningful. The correlation among the items involved in the sclae proves 
that this tool measures only a single factor (Crocker & Algina, 1986; Bryman, 1999). In this context, in order to 
calculate the correlation among the items constituting the grading rating scale, content structure of the grading 
rating scale structure which was going to be used to score the materials had been defined. 

At first, unrotated principal components analysis was applied to the data gathered from the scale and conducted 
to examine construe validity of the scale. In the analysis done on 50 items, there were 10 factors whose 
eigenvalue was found to be 1 and higher. The variance that was revealed by the whole elements was 71.58 but 
the variance revealed by 50 elements, which standed under the factor was 39.33. Factor-structure coefficients 
equal to or greather than .35 of the items was selected. In the analysis done, 9 items below .35 were taken out of 
the scale and it was found out that 41 items were functioning well. 

An Unrotated Principal Components Analysis was done with these 41 items. As a result of the analysis, the 
factor number was decreased from 10 to 6 and the variance value of the first factor was increased to % 46.37. 
After warimax rotation, the first factor loading was 32.43. Bartlett’s Test Sphericity is found to be 3440,283 and 
Kaiser-Meyer-Oklin Measure of Sampling Adequacy value .906. The total variance explained by 6 factors was 
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found to be %66.67. The elements that remain functioning after the scale factor decoding has been categorized 
under 6 titles (groups) according to their relation with each other (Table 1). By gathering these items together, 
the researchers themselves defined which titles would be in rubric taking the items in the scale into 
consideration. According to this; the titles has been grouped under 6 categories. These are: (1) audio- 
visualelements, (2) content, (3) language and expression, (4) functioning, (5) form-shape and (6) colour. The 
items involved in these titles and their factor-structure coefficients are given in Table 1. 


Item 

No 

Items 

Factor 

loading 


1- Audio-Visual Items 


2 

The visual elements in the material should enrich the education (learning) process (picture, 
photograph, schema, icon, diagram...etc) 

,856 

3 

The audio elements in the material should enrich the education (learning) process. 

,825 

4 

The visual elements in the material should be in close interaction with each other. 

,797 

5 

The material should be supported with striking visual elements. 

,797 

6 

The material should involve suprising stimulus that will attract the learners’ attention and keep it 
alive. 

,746 

8 

The cognitive learning aids (icon, hint.. .etc) should be used adequately and in different levels. 

,744 

15 

Many more visual and audio elements should be used in the material. 

,433 

18 

The topic should be sufficient. Besides, the content should be presented with the elements that can 
cater to different sense organs. 

,871 

26 

Innumerable surprising elements which will not irritate the learners should be used. 

,618 

44 

Although ready-made elements are used in the material, their using objectives and functions should 
be new and useful. (They should be pedagogically suitable) 

,794 

60 

The use of each element shouldn’t be the imitation of another. 

,580 


2- Content 


24 

The content should be authentic but it should also be presented in different context. 

,625 

29 

New ideas should be involved in the material. 

,805 

32 

The material should consist of live sensory explanation. 

,574 

48 

in terms of creativity, the material should show that the person having prepared it has understood the 
issue in depth. 

,762 

51 

The content should be presented in various ways different from the others. 

,568 

50 

The content in the material should be rich and detailed. 

,412 

59 

The material should offer solution to at least one of the difficulties in the content area which it aims 
to teach. 

,656 

62 

There should be an extraordinary connection or relation among the issues in the material. 

,484 

63 

The synthesis of the content in the material should have been reached. 

,734 

12 

The material should be backed up (supported) by the examples which don’t exist in other materials. 

,588 


3- Language And Expression 


13 

Important items in the material should be emphasized in a different way. 

,715 

16 

The material should consist of fewer texts but more rich concepts that reflect the issue. 

,622 

36 

The writing style should be original. 

,396 

49 

The words should be used in a rich way. 

,472 


4- Operating (Functioning) And Mechanism 


14 

The material should provide the learners with lots of various learning opportunities. 

,807 

25 

The material should be designed in such a way that considers different learning preferences of the 
learners. 

,789 

27 

The material should make learning enjoyable. 

,792 

30 

The material should attract the attention of the target group and keep it alive. 

,857 

41 

The prepared material should serve for different aims and requirements. 

,647 

46 

The material should force the imaginative power. 

,674 

47 

The material should be the most suitable material in terms of learning. 

,708 

54 

The material should be handled in a way that will rescue the learners from various thinking patterns. 

,761 
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56 | The materials should be previously-used. _ | ,821 



5-Form 


7 

The material should be easily followed and understood by the learners. 

,761 

9 

The material should involve various design patterns (formal-informal balance or symmetrical or 
irregular balance) in terms of form. 

,623 

33 

In the material, the issues-elements should be presented by using wide variety of approaches- 
elements. 

,658 

45 

The material should put forward a new style. 

,533 

61 

Each single page shouldn’t seem like a repetition of the other. 

,403 


6-Color 


10 

In the material, primary subtractive and additive colours should be used together in a suitable way. 

,637 

20 

The figure and background colors should be used in a way that is unfamiliar and different from the 
colors used in other materials. 

,374 


As seen in Table 1, 18th item has the highest factor loading (.87) and the 20th item has the lowest factor loading 
(.37) the spectrum of the total expressions (data) in the sub-dimensions that are formed by the items in the scale 
is; There are totally 11 items in the sub-dimensions of audio-visual elements, 11 items in the sub-dimension of 
content, 4 items in the sub-dimension of language and expression, 9 items in the sub-dimension of functioning, 5 
items in the sub-dimension of form and 2 items in the sub-dimension of colour. 

After the Factor analysis, in the third phase, the sub-dimensions had been formed by benefitting from the items 
that remain functioning and the content of performance expressions that would take place within these sub¬ 
dimensions was defined. Later, by using all these data, a grading rating scale key was prepared. 

The principal element (e.g. number, practicability) which were emphasized in each item and which were 
important in terms of creativity was taken into consideration and they were changed into expressions of 
performance. Each title in the scale serves as the sub-dimension of the grading rating scale key. Besides, the 
performance expressions that are formed by all the materials in this dimension show the basic level of this 
dimension and the quality (characterictics) of a perfect study. For this reason, the level formed by the 
performance expressions resulting from the items that remain functioning after the factor analysis expresses the 
study with highest-level creativity (the most creative study). According to this; the levels that are involve in 
these 6 dimensions and; that are said to be the most and the least creative studies in each single dimension are 
written in Table 2. 


Table 2: Rubric 


Score 

1- Visual And Audio Elements 

5 

1- The visual and audio elements used in the material aren’t used before or even ready-made elements 
are used, they are adequate, different, new and useful in terms of use, objective and function. 

2- They involve numerous elements that enrich the learning process and that have mutual interaction. 

3- The content is presented with many surprising elements (visual and audio) that will attract the 
attention of the learners. 

4 

1- In the material ready-made materials is used. Flowever, each element is different, new and useful in 
terms of use, objective and function. 

2- The elements are interactive, adequate in number and they enrich the learning process. 

3- The visual and audio elements used to present the content attract the attention of the students and they 
are backed up with (supported) surprising elements from time to time. 

3 

1- Although not frequently, there are items that are used for the first time.the purpose of use, functions of 
the elements are beneficikal but not new. 

2- The elements are innumerable and they help the learning process. 

3- Although the visual and audio elements used in order to serve the content have an ordinary usage, 
they are interesting. 

2 

1- The elements that are used in the material are the materials that are used before. They are scarce in 
number, but they are functionally different and beneficial. 

2- Although not to a great extent, the elements make significant contributions to learning. 

3- The elements used to serve the content are insufficient to attract the attention of the learners and the 
material used to present the content isn’t coherent with the content. 

1 

1- The elements used in the material aren’t sufficient in number, there aren’t any differences and the 
benefit in using a material isn’t taken into concentration. 

2- The elements in the material do not contribute to learning process. 
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3- The material that is used to present the content is ordinary and it dosn’t reflect any relation and 
coherence. 


2- Content 

5 

1- The content is presented with authentic, rich in different contexts, detailed and extraordinary 
connections and new ideas. 

2- A content synthesis that shows a deeper understanding of the issue is formed it includes a lively 
sensory explanation with a different style and variety. 

3- The content comes up with solutions to more than one difficulty in the content field which the 
teaching has targeted. 

4 

1- The content is presented with highly detailed and new ideas. When presenting it, authentic and 
different contexts are taken into consideration. 

2- A content synthesis that shows the content is prepared and it includes diversity. 

3- The content comes up with a solution to at least one or two difficulties in the content field targeted by 
the teaching. 

3 

1- The content is presented with authentic, rich, detailed and different contexts. 

2- A synthesis that reflects the concepts of the issue is formed. 

3- The content comes up with a solution to at least one problem (difficulty) in the content field targeted 
by the teaching. 

2 

1- Although its presentation is detailed, it doesn’t involve authenticy, richness and new ideas. 

2- it shows that we have reached and understood the concept of the issue. 

3- The content comes up with a solution to at least one problem (difficulty) in the content field targeted 
by the teaching. 

1 

1- The content is devoit of detail, authenticity and richness, it cannot surpass the well-known form of it. 
(it is a repetition of its well-known form) 

2- The concept of the issue is ordinary. 

3- The content doesn’t come up with a solution to any problem (difficulty) in the content field targeted 
by the teaching. 


3- Language And Expression 

5 

1- The important parts in the material are emphasied in a different way with rich concepts and without 
any other materials. 

2- The meanings of the words are rich and the writing style is handled in a original and unique way. 

4 

1- The important parts in the material are emphasied in a different way with rich concepts and without 
any other materials. 

2- The meanings of the words are rich and the writing style is handled in a original and unique way. 

3 

1- Although all the important parts (points) in the material are emphasized, they are away from being 
rich and the use of the concepts are different from those in other materials. 

2- The meanings of the words are sometimes rich and the writing style is original from time to time. 

2 

1- The important points in the material are emphasized. The concepts are used carelessly. 

2- The words are used carelessly and the writing style is ordinary. 

1 

1- The concepts and the words are devoit of emphasis and they are ordinary. 

2- The words are used carelessly and the writing style is ordinary. 


4- Operating (Functioning) And Mechanism 

5 

1- The material is the most beneficial one in terms of learning, it makes learning enjoyable and it 
facilitates learning by attracting the attention of the target group. 

2- It gives answers to different needs and targets. It is practicle and provides different learning 
opportunities. 

3- It relieves the learners from thinking patterns and it has the mechanism that will foster imagination. 

4 

1- The material is suitable in tenns of learning and it attracts the attention of the target group. 

2- It provides answers to the needs and goals of the learners and it has the practicability, which will 
enable learning. 

3- It has the mechanism that will foster imagination and relieve the learners from thinking patterns. 

3 

1- The material is suitable in tenns of learning and it attracts the attention of the target group. 

2- It focuses on limited learning opportunity, it is practicle and it is far from taking the needs and 
expectations of the learners into consideration. 

3- Although it doesn’t aim to foster imagination, it has the meachanism to release the earners from 
specific thinking patterns. 

2 

1- Although the material is the beneficial in terms of learning, it doens’t make learning enjoyable and it 
doesn’t attract the attention of the target group. 

2- It is focuses on learning opportunity only with a single perspective. The needs and expectations are 
not taken into consideration. 
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3- Although it doesn’t aim to foster imagination, it has the meachanism to release the earners from 
specific thinking patterns. 

I 

1- The material cannot foster learning and it cannot attract the attention of the target group. 

2- It doesn’t focus on any learning opportunity. The needs and expectations are not taken into 
consideration. 

3- It has the meachanism that is suitable for the samiliar thinking patterns. 


5- Form (Shape) 

5 

1- The material has various design forms (such as fonnal-informal balance) that the learners can follow 
and understand easily. Each page is different from one another, (it has a unique quality) 

2- The issues and the elements are presented with different approaches. They have different styles.(they 
are unique) 

4 

1- The material has various design forms (such as fonnal-informal balance) that the learners can follow 
and understand easily. From time to time, the pages are like the repetition of each others. 

2- The issues and the elements are presented with different approaches. They have different styles.(they 
are unique) 

3 

1- The material has various design forms (such as fonnal-informal balance) that the learners can follow 
and understand easily. The pages are like the repetition of each others. 

2- The issues and the elements are presented with different approaches. They have similar styles with 
the others.(they are not unique) 

2 

1- The material has various design forms (such as formal-infonnal balance) that the learners may have 
difficulty in tracing. The pages are like the repetition of each others. 

2- The issues and the elements are presented with nearly similar approaches. Their styles are not 
different. 

1 

1- The material has various design forms (such as formal-informal balance) that the leamerscannot 
understand. The pages are copied (replicated) from other pages. 

2- The issues and the elements are always presented with similar approaches.they don’t have styles. 


6-Colour 

5 

Primary subtractive and additive colours used in the material are coherent (harmonic) all together and 
they have a completely different style of use. The colours of both figure and background are used in a 
familiar way which is different from the ones used in other materials. 

4 

The colours used in the material are all effective and they have a completely different style of use. The 
colours of both figure and background are used in a way that is generally unfamiliar. 

3 

The colours used in the material are all effective but they are used in a way that is used in other elements 
from time to time. The colours of both figure and background are used in an unfamiliar. 

2 

The colours used in the material are all effective but they are used in a way that is generally used in 
other elements. Although limited, the colours of both figure and background are used in an unfamiliar. 

1 

The colours used in the material are far from effectiveness. They have a familiar and ordinary usage. The 
colours of figure and background are ordinary and they have familiar usage. 


As it is seen in Table 2, the guideline is composed of 6 sub-dimensions with 5 different levels. 5 depict the most 
creative material, 4 creative materials, 3 mid-level creative meaterial, 2 low-level creative materials and 1 
uncreative material. 

In the 4 1 ' 1 phase of the study, it is tried to specify the consistency level among the scorers in order to determine 
reliability. In this context, 5 persons including 4 specialists in material development (2 researchers and 2 
specialists other than the researchers) and a specialist who only studies on creativity gave grades in 6 sub¬ 
dimensions to the materials prepared by students with 15 different caharacteristics in teaching technologies and 
material development course. The correlation among grades and Cronbach’s Alpha coefficients of each material 
was calculated with 6 sub-dimensions of the graduated rating key. As a result of the analysis done; 

• In the sub-dimension of visual and audio elements, the highest correlation was found to be .92 (second 
researcher and the Specialist Studying on the Field of Creativity [SSFC]) and the lowest one .79 (first researcher 
and the First Specialist in Material Development [1-SMD]) .The Cronbach’s Apha coefficient was .96. 

• In the sub-dimension of context, the highest correlation was found to be .93 (first and second reserachers); 
the lowest one 74 (first researcher and the SSFC) and the Cronbach’s Apha coefficient .94. 

• In the sub-dimension of language and expression, the highest correlation was found to be .88 (Second 
Researcher and SSFC) the lowest one .74 (Second Specialist in Material Development [2-SMD] and SSFC) and 
the Cronbach’s Alpha coefficient .91. 
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• In the sub-dimension of processing and mechanism, the highest correlation was found to be .91 (Second 
Researcher and SSFC), the lowest one .77 (First Researcher and 1-SMD) and the Cronbach’s Alpha coefficient 
.94. 

• In the sub-dimension of form, the highest correlation was found to be .89 (1-SMD and SSFC) and the lowest 
correlation was found to be .75 (first researcher and 2-SMD) and the Crobach’s Alpha coefficient .92. 

• In the sub-dimension of colour, the higest correlation was found to be .87 (first and second researcher) the 
lowest one .72 (2-SMD and Second Researcher) and the Cronbach’s Alpha coefficient .90. 

Although the correlation proves whether the relation among the grades was positive or meaningful, it doesn’t 
enable us to distinguish the quality of a study with 3 or 4 grades in a 5-graded key. Namely, while a scorer gives 
3 to a study, other scorer may give 4. In this situation, the correlation is high as well as there is a significant 
difference between a 3-graded and a 4-graded study in terms of creativity. 

In this context, 3 materials which has the highest (y= 4.6), middle (y=2.6) and the lowest (jc=\2) grades in the 
mean average used in the research and calculated as a result of the gradings of 5 scorers were selected and they 
were given grades by two independent researcher and the correlation percent was observed (Koretz et al., 1993; 
Burstein et al., 1996). 

As a result of the grading done, the grades that were given to the materials with three different characteristics 
and the correlation among the grades with the highest, middle and the lowest quality of creativity is given in 
Table 3. 


Table 3: The Correlation Grade Table in Terns of Two Scorers and Qualities. 


Quality 


Visual and 
Audio 
Elements 

Content 

Language 

and 

Expression 

Processing 

And 

Mechanism 

Form 

Colour 

The most 
Creative 

1 .Scorer 

5 

5 

5 

5 

4 

4 

2. Scorer 

5 

5 

5 

4 

5 

3 

Middle 

1. Scorer 

3 

4 

3 

3 

3 

2 

2. Scorer 

3 

3 

3 

4 

4 

3 

The 

lowest 

Creative 

1 .Scorer 

1 

1 

1 

1 

1 

1 

2. Scorer 

1 

1 

2 

1 

2 

1 


As it is seen in the Table 3, it may be said that the correlation between the basic quality of rubric and the 
dimensions of a creative material is high in general. Although the materials selected to be the most creative in 
the sub-dimension if colour gets the highest grade by two scorers, rubric cannot reach the specified grade level. 

A creative study possesses the quality of a creative product in terms of all its sub-dimensions. As each scale 
element defined with factor analysis is composed of related elements, the correlation between the grades related 
to each sub-dimension of a graduated rating guideline can be used as a scale of reliabilitity. In this context, the 
correlation between each sub-dimensions of the scale prepared was scrutinized. 

In the analysis done; the highest correlation was in the sub-dimensions of “visual and audio elements” and 
“processing and mechanism” with the value of .89, the lowest correlation was found as .77 in the sub-dimension 
of content and colour. The Cronbach’s Alpha value was calculated as .92. 

3. RESULT 

The increasing interest in performance evaluation in the evaluation of the student success is the urge to carry out 
this study. The necessity of evaluating with a reliable and valid evaluation instrument such as portfolio 
evaluation and project evaluation that shows improvement especially in recent years in evaluating the tasks and 
assignments constitutes the basic principle of the study. Keeping this idea in mind, it is aimed to develop a tool, 
which will enable us to grade the materials prepared by the students objectively in terms of creativity. 

In this context, a rubric was prepared in order to be used in the evaluation of the students’ materials in terms of 
creativity. First of all, the elements that define the basic qualifications which a creative material should possess, 
were found out by talking with the field specialists and scanning the literature. The qualities were transformed 
into 5 point-likert type scale and it was subjected to factor analysis in order to determine the basic qualities that 
could take place in the scale after it was applied to the specialists. 
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As a result of the factor analysis, it was found out that 41 items were closely related to the same factor (KMO = 
0,906 and the declared total variance 66,677) and they were collected in 6 sub-titles. In this way, the qualities of 
a well-qualified study were found out and the sub-dimensions of the scale were determined. 

Keeping the specified materials determined with factor analysis in mind, a grading rating scale was prepared. As 
these are creative materials, in the study, it is aimed to define the dimensions of a less creative material, in other 
words, its sub-dimensions. 

In the determination of the sub-dimensions, three researchers defined with what qualities the most creative; 
creative; mid-creative; the least creative material and the material, which was not creative, could be identified 
(based on the materials gathered from the scale) and then a rubric was prepared after the dimensions done by 
these three researchers were combined. In terms of the reliability of this key, the coefficient alpha of the 41-item 
scale was .90 and the correlation between grades above that was .93 at the top and the lowest correlation was 
found as .72. The correlation among the values gathered from the total grades that these 5 evaluators gave to 
each sub-dimension of the grading rating scale in terms of reliability is scrutinized. The lowest correlation in the 
analysis was .77. 

In such kind of grading rating scale, the generalizability of the rating guideline is important. (Koretz et al., 1993; 
Burstein et al, 1996) However, in this study, we put up only with the correlation among the sub-dimensions of 
the rubric as a scale of the generalizability of the students’ studies. In such studies, the grades and correlation 
that can be used as a scale is important in terms of generalizability (Koretz et al., 1993; Shavelson & Webb, 
1991; Burke, 1999). But in this study, the fact that there is only one sample of the students’ studies and there 
aren’t any grades related to the creative material pertaining to the sampling has prevented the analysis in this 
aspect. 

The prepared grading rating scale possesses the usable qualities as a reliable and valid tool in the sub-dimensions 
of audio- visual elements, content, language and expression, processing and mechanism, form and final colours 
in the evaluation process in terms of the teaching materials’ creativity. 
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